Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure

DR Roberts; V Bahn; S Ciuti; MS Boyce; J Elith; G Guillera-Arroita; S Hauenstein; JJ Lahoz-Monfort; B Schröder; W Thuiller; DI Warton; BA Wintle; F Hartig; CF Dormann

Journal article

Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure

DR Roberts, V Bahn, S Ciuti, MS Boyce, J Elith, G Guillera-Arroita, S Hauenstein, JJ Lahoz-Monfort, B Schröder, W Thuiller, DI Warton, BA Wintle, F Hartig, CF Dormann

Ecography | WILEY | Published : 2017

DOI: 10.1111/ecog.02881

Abstract

Ecological data often show temporal, spatial, hierarchical (random effects), or phylogenetic structure. Modern statistical approaches are increasingly accounting for such dependencies. However, when performing cross-validation, these structures are regularly ignored, resulting in serious underestimation of predictive error. One cause for the poor performance of uncorrected (random) cross-validation, noted often by modellers, are dependence structures in the data that persist as dependence structures in model residuals, violating the assumption of independence. Even more concerning, because often overlooked, is that structured data also provides ample opportunity for overfitting with non-caus..

View full abstract

University of Melbourne Researchers

Jane Elith Author

Gurutzeta Guillera-Arroita Author

Jose Lahoz-Monfort Author

Brendan Wintle Author

Related Projects (2)

Using species distribution models to make robust conservation decisions

Species distribution models inform numerous conservation decisions, from planning reserves and managing biological invasions to assessing cl..

Connections between imperfect detection and ecological inference

This project is designed to resolve whether or when it is important to account for imperfect detection when modelling communities of species..

Grants

Awarded by German Science Foundation

Awarded by DFG

Awarded by Australian Research Council

Funding Acknowledgements

DRR is supported by the Alexander von Humboldt Foundation through the German Federal Ministry of Education and Research. BS is supported by the German Science Foundation (grant no. SCHR1000/6-2). CFD acknowledges additional funding by the DFG (DO786/10-1). DIW and JE are supported by Australian Research Council Future Fellowships (grant no. FT120100501 and FT0991640). GGA is the recipient of a Discovery Early Career Research Award from the Australian Research Council (project DE160100904). The work of JJLM was supported by the Australian Research Council Discovery Project DP160101003. Collection of data used to build elk resource selection models (Box 2) was funded by the Alberta Conservation Association (ACA - Grant Eligible Conservation Fund; grants to SC and MSB), the Natural Sciences and Engineering Research Council of Canada (NSERC CRD; grants to MSB and postdoctoral fellowship to SC), and Shell Canada limited. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Author contributions - All authors conceived the idea for this study. DRR, FH, CFD, SC and VB designed the study, and DRR, SC and VB carried out the simulations and analyses. The initial draft was written by DRR. All authors contributed comments and improvements to the manuscript.